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FAREWELL TO THE TURING TEST? 


Sixty years after Alan Turing’s seminal article in which he introduced the Imitation Game, the discussion on the 
scientific value of the game is still lively and challenging. Nowadays the Imitation Game is called the Turing 
Test. A modern and more practical form of this test is the Loebner Prize contest, which will have had its 20" 
edition on October 23, 2010. 


It is safe to say that “passing the Turing Test” has served for sixty years as a paradigm for researchers in 
Artificial Intelligence, Cognitive Science, Philosophy, and Natural Language Processing. It is attractive to 
compare the outcome of the “thinking” process of a machine to the result of the thinking process of a human 
being. In the beginning (1950) such a machine was a one-processor engine and the contest was challenging for 
engineers and scientists. Gradually, the power of the machine increased and its storage capacities grew beyond 
the limits that are imposed by nature on human beings. (Here I admit that, in the future, technology may help 
human brains to augment the memory capabilities, but I disregard this development for the moment.) 


The question I would like to address here is the following: is the Turing Test still relevant to our researchers? 
We assume that some of our readers will answer this question by ‘yes’. In particular, the Editorial Board may be 
expected to give this answer since we publish in this issue a contribution by Maarten van Emden and André 
Vellino, titled From Chinese Room to Human Window. Humanising the title would lead to: “From Searle to 
Michie and back again, with Turing as interested spectator.” 


A fair Turing Test would be a test between equally capable entities. From 1950 to 1990 a test between a 
computer and a human being on their thinking capabilities in one or another area (or even in all areas of 
thinking) could be considered as a rather fair contest. However, since 1990 the information technology world 
has experienced accelerating progress in areas such as supercomputers, the Internet, multi-processor systems, 
and distributed multi-agent systems. Can we consider the Internet or a multi-agent system as an entity? If the 
answer is affirmative, then the next question is: can we consider such an entity as a suitable opponent for a 
human being in a Turing Test? 
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Recently, the Dutch journalist Bennie Mols discussed this topic in the Dutch daily paper NRC Handelsblad. In 
the article, the philosopher Johan van Benthem criticised the Turing Test by remarking that a network of 
computers is not the same as “a machine”. His argument was as follows. For a machine, we have a model (i.e., 
the Turing machine); for networks with many computers involved we lack an acceptable theoretical model. His 
second criticism was directed at the dichotomy of the outcome: it is either passing the Turing Test or not 
passing the Turing Test. There is no measure on the outcome, e.g., the machine scored reasonably well, or it 
achieved 40 per cent. 


From the perspective of Artificial Intelligence and Cognitive Science, there are many more criticisms on the 
value of the Turing Test with respect to the current status of the technology. We mention (1) reactions on 
environmental input and (2) non-verbal communication. 


In the same article, the Dutch scientist Frank van Harmelen states: “On the one hand, I believe that the Turing 
Test currently is an obstacle for scientific research to understand the very nature of artificial intelligence (...). 
On the other hand, I still consider a computer that passes the Turing Test as a fulfilment of the extreme dream of 
our research community, comparable to the extreme dream in physics on a Theory of Everything. We should 
keep that dream, but not take it as a guideline for our research.” 


In the world of chatter bots, conversational systems, robots, and androids, much discussion is devoted to the 
value of the Turing Test. In the 2006 conference celebrating 50 years of Artificial Intelligence, participants 
could fill in a questionnaire on the “passing of the Turing test” (i.e., computers are able to mimic human 
intelligence). Forty-one per cent of the respondents believed that passing the test never happen. Another forty- 
one per cent believed it to be possible (but that it would take over fifty years to happen). Apparently, Ray 
Kurzweil is an exception (cf. Mols, 2010). 


In the previous issue we discussed the notion of a paradigm shift with respect to minimax and MCTS. 
Simultaneously we were facing the fact that the Drosophila Melanogaster had lost its paradigm function. Your 


Editor believes that we are now at the point of saying Farewell to the Turing Test. 


Reference 
Mols, B. (2010). De computer kletst maar wat. NRC Handelsblad, Science, September 25 & 26, pp. 8-9. 


Jaap van den Herik 


ICGA Journal readers who are interested in information on our publications are referred to our website. A 
complete list of all articles, notes, and literature reviews published in the ICCA Journal and the ICGA 


Journal is accessible on the Internet at http://www.icga.org 


The credits of the photographs in this issue are to: Michael Babar, David Fotland, Jaap van den Herik, Shi-Jim 
Yen and many anonymous photographers. 


Remark 1: Some reports of the Kanazawa events are postponed to the December issue, among them the report 
on the First World Computer Software Championship. It was won by SHREDDER (7 out of 8). 


Remark 2: We are grateful to all sponsors of the Kanazawa events, they are listed below, and only mentioned 
once in the general report on the JAIST Computer Olympiad. They are: JAIST, Ishikawa Prefecture, Fujitsu, 
Komatsu, Hokkoku Bank, Matsusaki, NTTWEST, Gakken Education Publishing, I-O DATA, Tanaka & 
Shobundo Graphic Art, PFU, Foundation for JAIST, Hokuriku, ICGA, ChessBase, and TiCC. 


